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Abstract 

The federal government recently enacted its first school voucher program as a pilot project in 
the District of Columbia. To be eligible, students need to be entering grades K-12 and have a 
family income at or below 185 percent of the poverty level. Although a rigorous analysis of the 
Opportunity Scholarship Program’s impact on student achievement and other outcomes remains 
a prospect for the future, at this early point initial data exists regarding the families that are 
applying for the program and the students that are using and not using the voucher when 
offered. Here we present a preliminary analysis of those data. We find that program applicants 
are somewhat disadvantaged relative to non-applicants regarding educational characteristics 
and family income, and are more likely to be African American, than non-applicants. The fact 
that the program is means-tested appears to be central to the finding that it is reaching a more 
disadvantaged population of students. When we examine all students that received a voucher 
award, and compare the group of voucher users with the group of voucher decliners, we find two 
significant differences. First, scholarship users are educationally advantaged in important ways 
relative to scholarship decliners. They are much less likely to have learning or physical 
disabilities, and younger scholarship users evidence somewhat higher test scores than non-users 
in similar grades. Second, we find that scholarship non-users are more likely to report that their 
existing school has various specialized educational programs and more extensive facilities. 
Although these results suggest some measure of selectivity in the group of actual program 
participants, the data do not indicate conclusively if that selectivity is a function of the decisions 
and behavior of participating private schools or the result of the rational decisions of consumers 
in a newly-expanded education market. 
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On January 23, 2004, President Bush signed the DC School Choice Incentive Act into 
law/ This landmark piece of legislation included $14 million in funding for what would come to 
be called the DC Opportunity Scholarship Program (OSP). The OSP is the first federally funded 
K-12 scholarship program in the country and is designed to provide approximately 1,700 low- 
income DC children with tuition scholarships worth up to $7,500 to cover the costs of attending 
participating K-12 nonpublic schools in the District. The pilot program is authorized to operate 
for five years and is being implemented by a group of non-profit organizations headed by the 
Washington Scholarship Fund. 

In addition to extending educational choices to an economically disadvantaged group of 
DC families, the OSP also provides the opportunity to learn more about what happens when 
more families have the opportunity and responsibility to choose a private school for their 
children. The U.S. Department of Education, through the Institute for Education Sciences, has 
contracted for the conduct of a rigorous experimental evaluation of the impact of the program on 
a number of student outcomes including student achievement. Here we provide information 
from a separate, independent assessment of the characteristics of the initial cohort of eligible 
families that appear to influence their decisions to (1) apply for the program and (2) use a 
scholarship to attend a private school of choice if awarded a scholarship. 

The information presented in this report was gleaned from data obtained from program 

applicants and the District of Columbia Public Schools during the initial year of program 

implementation in 2004. The authors are all members of the research consortium selected by the 

U.S. Department of Education, Institute for Education Sciences, to provide technical assistance 

to the program in the first year and conduct a rigorous, comprehensive, experimental impact 

* Title III of the District of Columbia Appropriations Act of 2004, Division C of HR 2673, 118 Stat. 1 17, D.C Code 
Sec. 38-1851.01. 
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evaluation of the program during its statutory 5 -year pilot period. Although the data that inform 
this particular analysis were all collected as part of that broader evaluation, and the initial results 
presented here are all drawn from the evaluation’s baseline report to Congress , the 
interpretations and conclusions presented here are solely the independent judgments of the 
authors and should not be ascribed to the official evaluation, the U.S. Department of Education, 
or any of the institutions with which the authors are affiliated. 

Although this study is not part of the official evaluation of the Opportunity Scholarship 
Program (OSP), it does provide a concise and revealing description of what sorts of eligible 
families initially applied for the program, did not apply for the program, and used a scholarship if 
offered. Thus, this analysis speaks to important policy questions of the extent to which the 
program is reaching its target population and what potential barriers might be preventing targeted 
families from fully participating in the nation’s first federally-sponsored school voucher 
program. 

As a preliminary analysis, this study has important shortcomings. First, it examines only 

the initial cohort of participating families recruited in April and May of 2004. Since then, a large 

second cohort of eligible participants has joined the program, filling it to capacity and generating 

-2 

a sizable randomized control group. The pattern of who chooses and uses from that second 
cohort of participants may be substantially different from the pattern uncovered for the first 
cohort in this paper. Second, our preliminary analysis merely involves comparing the means of 
various relevant populations of eligible program applicants, non-applicants, scholarship users 

^ Patrick Wolf, Babette Gutmann, Nada Eissa, Michael Puma, and Marsha Silverberg, Evaluation of the DC 
Opportunity Scholarship Program: First Year Report on Participation U.S. Department of Education, National 
Center for Education Evaluation and Regional Assistance (Washington, DC: U.S. Government Printing Office, 
2005). 

^ Patrick Wolf, Babette Gutmann, Michael Puma, and Marsha Silverberg, Evaluation of the DC Opportunity 
Scholarship Program: Second Year Report on Participation, U.S. Department of Education, Institute of Education 
Sciences (Washington, DC: U.S. Government Printing Office, 2006). 
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and non-users. Once we have collected longitudinal data on program participation, we will be 
able to conduct more sophisticated statistical analyses of the factors that influence persistence in 
the program."^ 



THEORY AND PREVIOUS RESEARCH 

In the 50 years since economist Milton Friedman published “The Role of Government in 
Education”^ scholars and policy makers have been debating how parental choice through market 
mechanisms can and does operate in education. Market “optimists” argue that education is a 
service that can be produced under a variety of arrangements and that parents are natural 
education consumers.^ Market “pessimists” argue that education is a public good best produced 
in government-run schools, and that school choice programs are prone to “market failure” 
because advantaged families are more likely than disadvantaged ones to have the motivation, 
resources, and experience to choose schools effectively. 

Fortunately, researchers have begun to study how market mechanisms like school choice 
work in practice, shedding light on a number of academic disputes and policy questions. Still, 
much remains to be learned about how low-income families respond to the new opportunities 

o 

made available to them through school choice programs. Moreover, how school choice 



See, for example, William G. Howell, “Dynamic Selection Effects in Means-Tested, Urban School Voucher 
Programs,” Journal of Policy Analysis and Management, 23:2 (2004). 

^ Milton Friedman, “The Role of Government in Education” in Robert Solo (ed.). Economics and the Public Interest 
(Rutgers, NJ: Rutgers University Press, 1955). 

^ See Milton Friedman, Capitalism and Freedom (Chicago: University of Chicago Press, 1962); John E. Chubb and 
Terry M. Moe, Politics, Markets, and America’s Schools (Washington: Brookings, 1990). 

^ See John Dewey, Democracy and Education (New York: Macmillan, 1916); Amy Gutmann, Democratic 
Education (Princeton, NJ: Princeton University Press, 1987); Helen F. Ladd, Market-Based Reforms in Urban 
Education (Washington, DC: Economic Policy Institute, 2002); Henry L. Levin, “Educational Vouchers: 
Effectiveness, Choice, and Costs,” Journal of Policy Analysis and Management. 17, (June 1998). 

* See, for example, David E. Campbell, Martin R. West, and Paul E. Peterson, “Participation in a National Means- 
Tested School Voucher Program,” Journal of Policy Analysis and Management, 24:3 (2005); Edward Fisk and 
Helen Ladd, When School Compete: A Cautionary Tale (Washington: Brookings, 2000); Brian P. Gill, P. Michael 
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programs are designed, and the real-world context in which they are implemented, appears to 
strongly influence their success.^ 

Our purpose in examining the characteristics of parents and students who are 
participating in the OSP is precisely to better understand the reality within which this new school 
choice program is operating. The OSP, therefore, offers a rare contemporary opportunity to 
examine prominent theories and advance research on what kinds of parents and students avail 
themselves of school choice. 



Why are School Choice Programs Important? 

The OSP is a parental school choice program targeted to low-income families living in 
the District of Columbia. It permits parents to send their children to private schools, of their 
choosing, at public expense. School choice is not limited to places with K-12 scholarship 
programs, such as Milwaukee, DC, and the states of Ohio, Florida, and Utah. As Elmore and 
Fuller point out, “Choice is everywhere in American education. It is manifest in the residential 
choices made by families with school-age children; it is capitalized in the housing prices found 
in neighborhoods.”^*^ Since most school children are assigned to a public school based on where 
they live, housing markets play a very important role in American education, so much so that 



Timpane, Karen E. Ross, and Dominic J. Brewer, Rhetoric Versus Reality: What We Know and What We Need to 
Know About Vouchers and Charter Schools (Santa Monica, CA: RAND, 2001). 

® Jeffrey R. Henig, Rethinking School Choice: Limits of the Market Metaphor (Princeton, NJ: Princeton University 
Press, 1994), p. 21; Bruce Fuller, Richard F. Elmore, and Gary Orfield, “Policy Making in the Dark: Illuminating the 
School Choice Debate,” in Bruce Fuller and Richard F. Elmore (eds.). Who Chooses? Who Loses? Culture, 
Institutions, and the Unequal Effects of School Choice (New York: Teachers College Press, 1996), pp. 12-13; 
Richard F. Elmore and Bruce Fuller, “Empirical Research on Educational Choice: What are the Implications for 
Policy-Makers?” in Bruce Fuller and Richard F. Elmore (eds.). Who Chooses? Who Loses? Culture, Institutions, 
and the Unequal Effects of School Choice. (New York: Teachers College Press, 1996), p. 200; James Harvey and 
Lydia Rainey, Doing Choice Right: Proceedings of a Meeting on Communities and Choice (Center on Reinventing 
Public Education: University of Washington, 2004) p. 10. 

Richard F. Elmore and Bruce Fuller, “Empirical Research on Educational Choice: What are the Implications for 
Policy-Makers?” in Bruce Fuller and Richard F. Elmore (eds.). Who Chooses? Who Loses? Culture, Institutions, 
and the Unequal Effects of School Choice (New York: Teachers College Press, 1996), p. 187. 
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Patrick Wolf has argued: “School choice and residential assignment are both market-based 
methods for allocating students to schools.”" 

As an alternative to school assignment by residence, school choice programs hold the 
prospect of correcting some of the “market failures” associated with a reliance on real estate 
markets to match students with schools. After all, low-income families may not view their 
neighborhood public school as a desirable place for their children to be educated, and they tend 
to lack access to private schooling or high-priced residential areas with more attractive schools. 
However, the extent to which school choice programs operate democratically and effectively to 
place students in appropriate schools depends heavily on the willingness and ability of 
participating families to operate as effective education consumers. Thus, the first issue we will 
examine in this paper is the extent to which applicants to the OSP are representative of the larger 
population of DC students. 

How Selective are the Families that Actually Use Scholarships? 

A debate continues to rage in the policy world regarding the extent to which school 
choice programs will primarily serve the proverbial “cream of the crop” from among scholarship 
recipients. Theoretical works by a number of prominent academics have predicted that cream- 
skimming will dominate school choice programs. Other theorists argue that disadvantaged 
families will be more likely to avail themselves of new school choice programs, since their 



Patrick J. Wolf, “Comment on ‘School Choice: How an Abstract Idea Became a Political Reality,’” in Diane 
Ravitch (ed.). The 2004 Brookings Papers on Education Policy (Washington: Brookings, 2005), p. 164. 

See, for example, Albert O. Hirshman, Exit, Voice, and Loyalty (Cambridge: Harvard University Press, 1970); 
Bruce Fuller and Richard F. Elmore (eds.). Who Chooses? Who Loses? Culture, Institutions, and the Unequal 
Effects of School Choice (New York: Teachers College Press, 1996); Alan Wolfe (ed.), School Choice: The Moral 
Debate (Princeton: Princeton University Press, 2003). 
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schooling options are the most constrained by systems of assigning students to schools based on 
residence/^ 

In general, the results of previous studies suggest that the selectivity of a choice program 
depends heavily on how it is designed and implemented. Hamilton and Guin conclude that 
“Parental choice is obviously constrained by the options available to them, and in many cases the 
options are quite limited.”^"^ When parents have a variety of schooling choices and choice 
schools have a limited to select their choice students, parents are more likely to find a school that 
is a good fit for their child’s particular needs. When choices are limited and the schools 
themselves make the admission decision, the chances of a desirable placement can be 
significantly reduced. For example, Fiske and Ladd report that not all parents in New Zealand’s 
universal choice program had viable choices, given a shortage of alternatives and out-of-pocket 
expenses required to enroll in desired schools. Since oversubscribed schools were allowed to 
choose their students, “Parental choice, in short, gave way to school choice.”^^ 

When school choice programs are limited to low-income families, as is the case with the 
OSP, previous research indicates that most participants are highly disadvantaged. For example, 
the official evaluators of the Milwaukee and Cleveland school voucher programs concluded that 
those means-tested choice programs were reaching the disadvantaged populations of families 
intended by policy makers. Evaluations of voucher-like private scholarship programs have 
reported some evidence that the educational resources of eligible families — such as family 
income, mother’s education, and student’s prior academic achievement — appear to only 

See for example Caroline M. Hoxby, “Introduction,” in Caroline M. Hoxby (ed.), The Economics of School 
Choice (Chicago: University of Chicago Press, 2003). 

Hamilton and Guin, “The Demand Side of School Choice. . ., p. 20. 

Fiske and Ladd, When Schools Compete..., pp. 8-9. 

John F. Witte, The Market Approach to Education (Princeton: Princeton University Press, 2000), pp. 60-61, 67; 
Kim K. Metcalf, “Evaluation of the Cleveland Scholarship and Tutoring Program” (Bloomington: Indiana Center for 
Evaluation, University of Indiana, 1999), p. 14. 
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modestly and inconsistently influence their ability to use a scholarship. In the American 
context, race is often a very important consideration when examining participation in an 
education program and previous analyses of voucher programs in the U.S. have reached 
somewhat inconsistent findings regarding the extent to which race is a factor in influencing 

I S 

scholarship use. 

Probably the toughest test for the selectively of voucher programs is the extent to which 
they enroll students with education-related disabilities. Particular private schools have long 
served students with disabilities and some, such as the residential schools for hearing impaired 
students that are present in many states, are designed to serve such a clientele exclusively. Urban 
public school districts regularly contract with private schools to educate students with disabilities 
at public expense - a practice generally referred to as “non-public placements” - and America’s 
largest school voucher program is Florida’s McKay Scholarship Program for students with 
disability-related educational needs. However, private schools are not legally required to enroll 
or accommodate students with disabilities of a degree or kind that they are ill-equipped to serve 
effectively, and most of the urban private schools that participate in voucher programs are low- 
tuition religious schools that lack the resources and staff to teach students with more than modest 
educational disabilities. Thus, the second question we will explore is the extent to which such 
factors as race, income, parental education, prior academic achievement, and special education 
status affect scholarship usage in the OSP. 



William G. Howell and Paul E. Peterson, with Patrick J. Wolf and David E. Campbell, The Education Gap: 
Vouchers and Urban Schools (Washington: Brookings, 2002), pp. 56-89. 

** Campbell et ah, “Participation in a National, Means-Tested School Voucher Program. . Howell, “Dynamic 
Selection Effects. . . 

Jay P. Greene and Greg Forster, Vouchers for Special Education Students: An Evaluation of Elorida’ s McKay 
Scholarship Program, Manhattan Institute for Policy Research, Civic Report No. 38 (New York, June 2003). 
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What Motivates Families to Participate? 

If scholarship users tend to differ from non-users, is it because users are more motivated 
to initiate an educational change than are non-users? There is general agreement that parents 
seek educational choices beyond their assigned neighborhood public school because they are 
looking for a better educational alternative for their children. Fuller and his colleagues conclude 
that, “When benefits are targeted to low-income families, many parents do actively choose a 
school that they believe better fits their educational agenda than does the neighborhood 
school.” As Henig writes: 

Even under the best of circumstances, the neighborhood public school will not 

adequately serve the needs of every neighborhood child. This can be due to the 

particular characteristics of the child, the particular limitations of the school, or a 

2 1 

simple lack of fit between one and the other. 

There is some disagreement, however, regarding whether choice families tend to be 
primarily running away from a bad schooling situation or running toward a good one. Low- 
income parents and students in urban neighborhood public schools frequently report disturbing 
levels of violence in their schools, and resulting concerns about student safety. At the same 
time, a survey of the literature on why parents choose schools suggests that a better educational 
program, a religious educational environment, and a better match between the cultural values of 
the family and school are often motivators for choosing. Obviously, school choosers are 
looking for a change. Whether they are mainly seeking to escape from a bad situation or are 
simply interested in a distinct school that might be a better fit for their child remains an open 



Fuller et al, “Policy Making in the Dark. . p. 9. 

Henig, Rethinking School Choice..., p.206. 

Wells, “African-American Students’ View of School Choice...,’’ pp. 43, 45; Wolf et al. Evaluation of the DC 
Opportunity Scholarship Program. . ., p. C-5. William G. Howell and Paul E. Peterson, with Patrick J. Wolf and 
David E. Campbell, The Education Gap: Vouchers and Urban Schools (Washington: Brookings, 2002), pp. 110- 
112 . 

Valerie Martinez, Kenneth Godwin, and Erank R. Kemerer, “Public School Choice in San Antonio: Who Chooses 
and with What Effects?” In Bruce Euller and Richard E. Elmore (eds.), Who Chooses? Who Loses? Culture, 
Institutions, and the Unequal Effects of School Choice (New York: Teachers College Press, 1996), p. 54. 
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question. Thus, the third question we will explore in this paper is the extent to which families 
who are concerned about the safety or program offerings of their previous public schools are 
more likely to use a scholarship when offered. 

RESEARCH DESIGN AND DATA 

The conceptual foundation for our analysis is quite simple. A defined population of 
families was offered the opportunity to apply to the OSP. To what extent and regarding what 
factors did the sub-group of program applicants differ from the sub-group of non-applicants? In 
the initial year of the program, 1,366 eligible applicants were offered scholarships. A total of 
1 ,027 scholarship recipients “took up” or used their scholarship to enroll in a private school of 
choice as of September 10 of 2004, leaving 339 initial scholarship non-users or “decliners.” To 
what extent and regarding what factors did the sub-group of scholarship users differ from the 
sub-group of scholarship decliners? For all of these comparisons, simple two-tailed t-tests are 
used to identify whether group differences are statistically significant at or beyond the 95 percent 
confidence level. 

Our data for this analysis come from two sources. The comparison of program applicants 
to non-applicants is based on information in the 2003-04 Accountability Student Database 
provided by the DCPS Office of Communications and Public Information. The DCPS 
accountability database includes demographic information on 72,511 students in DCPS 
traditional and public charter schools, of which 1,077 could be definitively identified as program 
applicants. The group of applicants successfully matched in the DCPS database represents 80 
percent of all public school applicants. The DCPS database contains variables for student test 
scores, enrollment in the federal lunch program, grade, race, gender, and whether or not the 
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student has an Individualized Education Plan as a result of special educational needs linked to a 
disability. This part of the analysis necessarily excludes all 515 initial private school eligible 
applicants to the program, since no aggregate data are available regarding private school non- 
applicants. 

Test scores are a crucial element of most school choice analyses, even if the question at 
hand is not program impact but the degree of selectivity of program clientele. The accountability 
database contains the results of DCPS administration of the Stanford Achievement Test, Version 
9 (SAT-9) to all students in grades 3 through 11, and some students in grades 1 and 2, in the 
spring of 2004. These tests covered the areas of reading and mathematics, with results expressed 
in terms of National Percentile Ranks (NPRs). The NPR for a particular student test score 
describes the percentage of students who scored below that level on a nationally normed test for 
the specific grade level, subject, and testing period that applies to the student score. Expressing 
test-score norms in this way makes them approximately comparable across grade levels. 

Since our sample of program applicants matched in the DCPS database does not include 
20 percent of public school applicants, we conducted an additional analysis to test for sample 
bias. This sample bias analysis draws from the second database used in this paper - the 2004 
Program Database. The Program Database includes data on all applicants to the program in the 
first year, taken from the program application form and baseline survey. Since all public school 
applicants are in the program database, 80 percent are in both the program and accountability 
database, and 20 percent are only in the program database, a comparison of the characteristics of 
the subgroup that is in both databases with those of the subgroup only in the program database 



For example, the score average of nearly 47 NPRs for all K to 5 applicants in reading is about 3 percentile points below the 
national median for those grades in reading, as approximately 47 percent of students across the country who took the reading 
test last spring in those grades scored lower than the average for the program applicants. 
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provides information about possible bias in our “applicants versus non-applicants” analysis 
which necessarily draws exclusively from the accountability database. 

We compared the characteristics of matched versus unmatched applicants on 20 factors 
associated with the amount and sources of family income, parent marital status, and mother’s 
level of educational attainment. The two samples differed significantly on only one factor - 
average amount of child support received. One might expect a comparison of two samples to 
differ significantly on one of 20 factors merely by chance. Thus, this high level of comparability 
between the matched and unmatched groups of public school applicants suggests that the 
matched sample which is used in the analysis is representative of the population of the all public 
school applicants. 

The program database used to disconfirm the presence of sample bias in the 
accountability database also is used in this paper to analyze the selectivity of voucher users 
compared with voucher decliners. This database includes observations on all 1,366 students 
awarded scholarships in the first year of the program, of which 1,027 used the scholarship - a 
moderately-high usage rate of 75 percent - and 339 declined or failed to use. Although we 
cannot perfectly distinguish the non-users who merely decided not to use a scholarship from 
those who tried to use one but failed to gain access to a preferred private school, we can get some 
sense of the likely degree of exclusionary behavior on the part of participating private schools by 
the level of exclusivity of the group of scholarship users relative to non-users. 
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ANALYSIS AND RESULTS 

Our analysis proceeds in two stages. First we examine program applicants who were 
attending public schools at the time of application and compare them to their public school peers 
who did not apply for the program. Due to the limitations in the variables contained in the DCPS 
accountability database, this first set of comparisons will focus on a small set of educational 
characteristics and background factors. The second stage of the analysis will draw upon 
extensive data about all scholarship recipients and address the question of what factors appear to 
distinguish scholarship users from non-users. 

Who Applied to the Scholarship Program? 

The first set of question we explore is whether program applicants differ from non- 
applicants, and what that might suggest about the motivations of parents and the broader impaet 
of the program on the DC educational landscape. Here we draw from the DCPS accountability 
database to compare program applicants entering grades K-12 with DCPS non-applicants in 
similar grades. The grade-restriction on the data reduces the total number of observations to 
67,945. 

The non-applicant sample in this analysis includes DCPS students ineligible for the 
program due to family incomes above the statutory limit of 1 85 percent of the poverty line. 

Later we will conduct a similar analysis restricted to students that were income-eligible for the 
program. However, policy design determines who is and who is not eligible for school voucher 
programs, and it is useful to assess the extent to which the design of this particular school choice 
program may have mitigated the selectivity of the group of program applicants. 
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Table 1. Characteristics of DC Public School Students, Program Applicants 
and Non-applicants: Spring 2004 



Characteristic 


Applicants 


DCPS 

Sample 


Difference 


Baseline Test Scores^ 








Average Reading Percentile 


41.5 


40.1 


1.4 


Percent missing 


26 


26 




Average Mathematics Percentile 


47.2 


46.3 


1.0 


Percent missing 


25 


25 




Percent in Special Education 


16 


14 


2* 


Percent missing 


24 


23 




Percent, by Race 








African American 


92 


85 




Hispanic 


6 


9 




Other race^ 


2 


6 




Percent missing 


1 


1 




Percent, by Gender 








Female 


51 


50 


0 


Percent missing 


0 


1 




Percent Participating in Free/Reduced- 








price Lunch Program 


85 


68 




Percent missing 


2 


2 




Sample size 


1,077 


66,868 





* Statistically significant at the 95 percent confidence level. 

** Statistically significant at the 99 percent confidence level. 



* Test-score results are in terms of National Percentile Ranks, with 50 as the median score. 

^ “Other race” includes students who were identified as white, Asian, American Indian, or Alaska Native. 

NOTE: Detail may not sum to totals because of rounding. Applicant sample includes all applicants identified in the DCPS 

database, regardless of whether or not they participated in the accountability testing. 

SOURCE: Accountability testing database for District of Columbia public and charter schools, DCPS 
Office of Communications and Public Information. 
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The data indicate that program applicants are somewhat disadvantaged educationally and 
economically, and more likely to be African American, compared with non-applicants (Table 1). 
Applicants and non-applicants are similar, on average, regarding both their reading and math test 
scores at baseline; however, 16 percent of the applicant sample is comprised of students in 
special education compared with only 14 percent of the non-applicant sample. Applicants are 
more likely than non-applicants to be African American and less likely to be Hispanic or of 
another race - in this case, generally white or Asian. The two groups are closely matched 
regarding student gender; however, applicants are much more likely than non-applicants to be 
participants in the federal lunch program for low-income students. 
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Table 2. Characteristics of DC Public School Federal Lunch Program Students, 
Program Applicants and Eligible Non-applicants: Spring 2004 



Characteristic 


Applicants 


DCPS 

Sample 


Difference 


Baseline Test Scores* 








Average Reading Percentile 


40.2 


36.4 


3 9 ^^ 


Percent missing 


25 


26 




Average Mathematics Percentile 


46.7 


43.0 


3 7 ^^ 


Percent missing 


25 


24 




Percent in Special Education 


17 


15 


1 


Percent missing 


24 


22 




Percent, by Race 








African American 


93 


88 


5** 


Hispanic 


6 


10 


-5** 


Other race^ 


2 


2 


0 


Percent missing 


0 


0 




Percent, by Gender 








Female 


51 


51 


0 


Percent missing 


0 


0 




Percent Participating in Eree/Reduced- 








price Lunch Program 


100 


100 


0 


Percent missing 


0 


0 




Sample size 


894 


44,740 





* Statistically significant at the 95 percent confidence level. 

** Statistically significant at the 99 percent confidence level. 



* Test-score results are in terms of National Percentile Ranks, with 50 as the median score. 

^ “Other race” includes students who were identified as white, Asian, American Indian, or Alaska Native. 

NOTE: Detail may not sum to totals because of rounding. Applicant sample includes all eligible applicants identified in the 

DCPS database that were participating in the free and reduced price lunch program. 

SOURCE: Accountability testing database for District of Columbia public and charter schools, DCPS Office of 
Communications and Public Information. 



July 6, 2006 



17 









Education Working Paper Archive 



The picture changes in important ways when the two samples are limited to students 
participating in the federal lunch program. The lunch program was selected as an inclusion 
screen because its income restriction closely matches the income ceiling for eligibility for the 
voucher program. Limiting the comparison groups to students in the lunch program has the 
effect of reducing the size of the public school applicant group to 894 students and the size of the 

'VC 

non-applicant sample to 44,740. 

When we look only at income-eligible students that either did or did not apply for the 
program, the program applicants no longer appear to be disadvantaged overall relative to non- 
applicants (Table 2). The lunch program restriction automatically equalizes the two groups on 
our income measure, and the two groups remain similar regarding their gender composition. 
However, the race differences across the groups change somewhat. Hispanic students remain 
somewhat under-represented in the group of program applicants, and African Americans 
somewhat over-represented, but the proportion of students in the other race category (i.e. white 
or Asian) becomes similar across the comparison groups. Among income-eligible students, the 
percentages of students in both groups who are in special education are statistically similar. 
Importantly, the baseline test-scores of program applicants compared with income-eligible non- 
applicants average nearly four percentile points higher in both reading and math. 



Although all 1,077 applicants matched in the DCPS database were confirmed to be income-eligible for the 
voucher program and thus almost certainly also eligible for the lunch program, it is well known that not all families 
eligible for means-tested programs such as the federal lunch program actually enroll in the program. To be included 
in the sample for this analysis, students had to actually be participating in the federal lunch program at baseline. 
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Table 3. Percent of Students Entering Various Grades, Eligible Applicants 
and Eligible Non- Applicants: Spring 2004 



Grade Level' 


Applicants 


Non- 

Applicants 


Difference 


Kindergarten -1 


20 


11 




2-3 


18 


17 


1 


4-5 


18 


17 


1 


6-7 


19 


18 


1 


8-9 


15 


15 


0 


10-12 


10 


21 


-11* 


Percent missing 


0 


0 




Sample size 


1,848 


71,434 





* Significantly different from the average of the other subgroups at the 95 percent confidence level. 

** Significantly different from the average of the other subgroups at the 99 percent confidence level. 

* Grade levels were combined because cell sizes were too small to report separately by grade. 

SOURCES: Eligible Applicant Database; DCPS Accountability Testing Database. 

Finally, we conduct comparisons of applicants with income-eligible non-applicants 
within relevant grade groupings. Applicants were much more likely than non-applicants to be 
entering the early elementary grades of K-1 and much less likely to be entering high school at the 
time of application in the spring of 2004 (Table 3). Thus, some of the differences between 
applicants and program-eligible non-applicants might be a result of this compositional difference 
- younger students are a greater proportion of the applicant group and may be different than 
older students in certain respects. 

Our analysis largely bears out that expectation (Table 4). The modest test-score 
advantage of applicants relative to program-eligible non-applicants is confined to students in the 
younger elementary grades. Applicants entering junior high or high school are similar to non- 
applicants regarding their average baseline test scores. The group of junior high applicants also 
is similar to the group of junior high non-applicants regarding racial demographics, though the 
early elementary and high school groups of applicants remain more likely to be African 
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American than grade-comparable non-applicants. The groups of program applicants in all three 
grade-bands are similar to their respective grade-band groups of non-applicants regarding special 
education status, gender, and, by analytic design, participation in the federal lunch program. 

In summary, the extent to which the initial applicants to the Opportunity Scholarship 
Program appear to be selective regarding educational, social, and economic factors depends on 
the composition of the group of non-applicants with which we compare them. Compared to all 
DCPS students, program applicants appear to be somewhat disadvantaged. When the 
comparison is limited to program-eligible non-applicants, the applicants are somewhat 
advantaged educationally and remain more likely to be African American. When even finer 
comparisons are made, between program applicants and program-eligible non-applicants within 
defined grade-ranges, we see that junior high applicants are statistically similar to non- 
applicants, early elementary applicants are somewhat advantaged educationally and more likely 
to be African American, and high school applicants are more likely to be African American. 

Who Uses an Opportunity Scholarship? 

In the next stage of our analysis, we examine the characteristics of scholarship users and 
non-users. The sample includes all 1,366 students offered scholarships during the initial year of 
program implementation, including 216 eligible applicants who were attending private schools at 
the time of application. We compare the two groups - users versus non-users — based on four 
sets of factors: educational characteristics of students, family background, student ethnicity and 
gender, and previous educational environment. 

The students who used their scholarships differed from non-users on some but not all key 
educational characteristics (Table 5). In the early elementary school grades of K-5, on average. 
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users outperformed non-users in reading by nearly 9 NPR and in math by nearly 10 NPR. Users 
in the junior high and high school grades did not differ significantly from non-users in those 
grades regarding reading and math baseline scores. Students whose parents said that they had a 
learning or physical disability were much less likely to use a scholarship if offered. Students 
with disabilities comprised just nine percent of the user group but 29 percent of the non-user 
group. Students were much more likely to use a scholarship if entering the early grades of K-3 
but were much less likely to use a scholarship if entering the junior or high school grades of 6- 
12 . 
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Table 4. Characteristics of DC Public School Federal Lunch Program Students by Grade Band, Program Applicants and Non- 
applicants: Spring 2004 





Grades K-5 


Grades 6-8 


Grades 9-12 






DCPS 






DCPS 






DCPS 




Characteristic 


Applicants 


Sample 


Difference 


Applicants 


Sample 


Difference 


Applicants 


Sample 


Difference 


Baseline Test Scores* 




















Average Reading Percentile 


46.5 


42.6 


4.0* 


38.0 


36.4 


1.5 


29.7 


28.8 


0.8 


Percent missing 


42 


44 




3 


5 




3 


14 




Average Mathematics Percentile 


51.2 


47.8 


3.5* 


43.8 


42.3 


1.5 


41.7 


37.9 


3.8 


Percent missing 


41 


42 




2 


5 




3 


13 




Percent in Special Education 


17 


13 


3 


18 


17 


1 


14 


16 


-2 


Percent missing 


41 


41 




2 


3 




2 


9 




Percent, by Race 




















African American 


94 


87 


7 ** 


88 


90 


-1 


95 


89 


6* 


Other race^ 


6 


13 


_ 7 ** 


12 


10 


1 


5 


11 


-6* 


Percent missing 


0 


0 




0 


0 




0 


1 




Percent, by Gender 




















Female 


52 


51 


1 


51 


51 


0 


46 


52 


-6 


Percent missing 


0 


0 




0 


0 




0 


1 




Percent Participating in Free/Reduced- 
price Lunch Program 


100 


100 


0 


100 


100 


0 


100 


100 


0 


Percent missing 


0 


0 




0 


0 




0 


0 




Sample size 


509 


20,893 




259 


12,364 




126 


11,483 





* Statistically significant at the 95 percent confidence level. 

** Statistically significant at the 99 percent confidence level. 

* Test-score results are in terms of National Percentile Ranks, with 50 as the median score. 

^ “Other race” includes students who were identified as Hispanic, white, Asian, American Indian, or Alaska Native. Because of small cell sizes, Hispanic was combined with 
other race in this table. 

NOTE: Detail may not sum to totals because of rounding. Applicant samples include all eligible applicants identified in the DCPS database that were participating in the free 

or reduced-price lunch program. 

SOURCE: Accountability testing database for District of Columbia public and charter schools, DCPS Office of Communications and Public Information. 
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Table 5. Student’s Educational Characteristics, Users and Nonusers: Spring 2004 



Characteristic 


Users 


Nonusers 


Difference 


Baseline Test Scores* 








Average Reading Percentile 








Grades K-5 


49.12 


40.30 


8.82** 


Grades 6-8 


39.82 


38.06 


1.76 


Grades 9-12 


30.00 


23.57 


6.43 


Percent missing 


56 


41 




Average Mathematics Percentile 








Grades K-5 


53.47 


43.75 


9.72** 


Grades 6-8 


44.91 


42.26 


2.65 


Grades 9-12 


40.30 


43.83 


-3.53 


Percent missing 


56 


40 




Percent with a Learning or Physical Disability 


9 


29 


-20** 


Percent missing 


6 


8 




Grade Level 








K-1 


27 


14 


13** 


2-3 


23 


18 


5* 


4-5 


22 


22 


0 


6-7 


15 


22 




8-9 


9 


15 


-6* 


10-12 


3 


10 




Percent missing 


0 


0 




Sample size 


1,027 


339 





* Statistically significant at the 95 percent confidence level. 

** Statistically significant at the 99 percent confidence level. 



Percentages may not sum to 100 due to rounding. 

* Test-score results are in terms of National Percentile Ranks, with 50 as the median score. 

SOURCE: Eligible Applicant Database. Test-scores obtained from the accountability testing database for District of 
Columbia public and charter schools, DCPS Office of Communications and Public Information. 



Regarding family background, users differed from non-users in some important respects 
but not others. There is little distinction between users and non-users regarding family income, 
as both groups represent populations that are about equally disadvantaged economically, with 
annual family incomes averaging less than $20,000. Mothers of scholarship users have slightly 
more education, averaging two-tenths of a year more formal education than the mothers of non- 
users. However, users are indistinguishable from non-users regarding the likelihood of their 
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mothers being employed full time or being married. One surprising initial finding regarding 
scholarship users is that they reported less residential stability, averaging 1.6 fewer years at their 
current residence than non-users. 
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Table 6. Family Background Information, Users and Non-users: Spring 2004 



Characteristic 


Users 


Nonusers 


Difference 


Percent of Students whose Family Income is 








Less than $5,000 


11 


11 


0 


$5,000-$10,999 


18 


24 


-6* 


$ll,000-$24,999 


43 


38 


5 


$25,000-$39,999 


25 


24 


1 


$40,000 or more 


3 


4 


0 


Average Family Income 


$18,652.49 


$17,840.70 


$811.79 


Percent missing 


0 


0 




Percent of Students whose Mothers Attained the 








Following Level of Education 








No high school diploma 


11 


19 




GED 


6 


6 


0 


High school diploma 


25 


25 


0 


Some college 


35 


30 


5 


Vocational (2 yr) degree or certificate 


15 


14 


1 


College graduate 


7 


4 


3 


Graduate degree 


1 


2 


-2* 


Percent missing 


11 


19 




Average Years of Mother’s Education 


12.73 


12.50 


.23* 


Percent missing 


11 


19 




Percent of Students with Mothers Employed Eull 


46 


44 


2 


Time 








Percent missing 


13 


23 




Percent of Students with Married Mothers 


19 


16 


2 


Percent missing 


10 


18 




Average Years at Current Residence 


6.11 


7.69 


-1.58** 


Percent missing 


2 


3 




Sample size 


1,027 


339 





* Statistically significant at the 95 percent confidence level. 

** Statistically significant at the 99 percent confidence level. 

SOURCE: Eligible Applicant Database. 



The user and non-user groups do not differ significantly regarding student race, ethnicity, 
or the primary language spoken in the home (Table 7). Although gender was not a factor in 
distinguishing applicants from non-applicants, it was a factor in shaping the decision to use a 
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scholarship, as the composition of the user group was 9 pereentage points more female than the 
composition of the decliner group. 



Table 7. Student’s Ethnicity and Gender, Users and Non-users: Spring 2004 



Characteristic 


Users 


Nonusers 


Difference 


Percent, by Race 








African American 


94 


96 


-2 


Other race' 


6 


4 


2 


Percent missing 


6 


6 




Percent Hispanic (any race) 


1 


1 


1 


Percent missing 


4 


1 




Percent, by Language Most Spoken in Home 








English 


93 


96 


-3 


Other language^ 


7 


4 


3 


Percent missing 


4 


4 




Percent, by Gender 








Female 


53 


45 




Percent missing 


1 


2 




Sample size 


1,027 


339 





* Statistically significant at the 95 percent confidence level. 

** Statistically significant at the 99 percent confidence level. 



* “Other race” includes respondents who were identified as white, Asian, American Indian, Alaska Native, Pacific 
Islander, or multiracial. Respondent classified as “multiracial” if more than one race category selected. 

^ “Other language” includes all other languages including Spanish and Amharic. 

NOTE: Detail may not sum to total because of rounding. 

SOURCE: Eligible Applicant Database. 

The previous educational behaviors and experiences of users and non-users might differ 
in meaningful ways. First, we consider levels of parental involvement in the child’s education 
(Table 8). We see that both groups of parents, scholarship users and non-users, reported being 
heavily involved in their child’s education at baseline. However, scholarship users were 
somewhat more likely than non-users to say that they helped their child with reading or math 
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work not related to homework, assisted with homework, and attended school activities with their 
child. 



Next, we analyze differences between users and non-users regarding problems at their 
child’s previous school. Parents whose children encountered disorder or danger regularly in their 
previous school are probably more likely to use a scholarship as a means to escape such an 
environment. We see some evidence that problems at the previous school were an extra 
motivator for scholarship users (Table 9). Scholarship users were somewhat more likely than 
non-users to report that drug distribution and drug and alcohol use were serious problems at their 
child’s previous school, though there was no difference between the two groups in reports of 
several other categories of danger and disruption. 

Table 8. Parental Involvement with Child’s Education, Users and Non-users: Spring 2004 



Characteristic 


Users 


Nonusers 


Difference 


Percent of Parents who Participated in the 
Following Activities with Their Child in the 
Past Month 








Discussed experiences at school 


99 


98 


0 


Helped with mathematics or reading not 
related to homework 


94 


90 




Worked on homework 


97 


92 


5** 


Worked on a school project 


86 


85 


1 


Attended school activities with child 


90 


85 


5* 


Average percent missing 


8 


6 




Sample size 


1,027 


339 





* Statistically significant at the 95 percent confidence level. 

** Statistically significant at the 99 percent confidence level. 

SOURCE: Eligible Applicant Database. 
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Table 9. Problems at Child’s School, Users and Nonusers: Spring 2004 



Characteristic 


Users 


Nonusers 


Difference 


Percent of Parents who Believe the 
Following Problems at School are 
Serious 

Fighting 


50 


46 


4 


Tardiness 


48 


48 


1 


Destruction of property 


35 


35 


1 


Tmancy 


34 


39 


-4 


Cheating 


26 


22 


4 


Guns or other weapons 


20 


15 


4 


Drug distribution 


16 


10 


6* 


Drug and alcohol use 


14 


10 


5* 


Average percent missing 


10 


9 




Sample size 


1,027 


339 





* Statistically significant at the 95 percent confidence level. 

** Statistically significant at the 99 percent confidence level. 

SOURCE: Eligible Applicant Database. 



Finally, we consider the extent to which non-users may have felt motivated to decline the 
scholarship and remain at their previous school due to the school’s facilities or programs. Here 
we see some of the largest and most consistent differences between users and non-users (Table 
10). Scholarship non-users consistently reported that their child’s previous school contained 
more extensive facilities and specialized programs than scholarship users. For example, 17 
percent more non-users than users reported that their child’s previous school had a special 
program for non-English speakers, 15 percent more said that special education programs were 
offered, 12 percent more reported that the school included a special program for advanced 
learners, and eight percent more said that individual tutors were available to students. The 
parents of scholarship non-users were more likely than those of scholarship users to report that 
their previous school had a gym, nurse’s office, cafeteria, prepared lunches, child counselors, and 
library. If parent reports are to be believed, the schools that scholarship users decided to leave 
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contained fewer special educational programs and facilities than the schools that non-users 
decided to remain in. 



Table 10. School Facilities and Homework, Users and Non-users: Spring 2004 



Characteristic 


Users 


Nonusers 


Difference 


Percent of Parents Reporting that Students have 
the Following Resources at Their School 








Special programs for non-English speakers 


37 


54 




Nurse’s office 


80 


89 




Prepared Lunches 


70 


77 


_ 7 ^ 


Cafeteria 


81 


90 




Special programs for advanced learners 


33 


46 


-12** 


Arts program 


67 


72 


-4 


Special education programs 


57 


73 


-15** 


Computer lab 


75 


79 


-5 


Gym 


58 


68 


-10** 


Child counselors 


78 


84 


_ 7 ^ 


Library 


80 


86 


-6* 


After-school program 


82 


78 


4 


Music program 


69 


72 


-4 


Individual tutors 


37 


45 


-8* 


Average Hours of Daily Homework 


.93 


.96 


-.04 


Percent missing 


9 


7 




Sample size 


1,027 


339 





* Statistically significant at the 95 percent confidence level. 

** Statistically significant at the 99 percent confidence level. 



SOURCE: Eligible Applicant Database. 



DISCUSSION 

This initial investigation into who chooses and who uses an Opportunity Scholarship in 
the District of Columbia offers an interesting, complex, though tentative glance at the 
characteristics, motivations, and possible obstacles faced by parents and students. Far from 
being a privileged sub-section of the District’s educational system, initial applicants coming 
from DC public schools were similar to non-applicants in their baseline test scores and more 
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likely to be enrolled in special education, African American, and participants in the federal lunch 
program for low-income students. 

The disadvantaged nature of the population attracted to the program appears to be largely 
a function of the policy decision to restrict program eligibility to students with family incomes at 
or below 185 percent of the poverty line. When the comparison is made with program-eligible 
non-applicants, applicants have slightly higher test scores and remain slightly more likely to be 
African American than similarly impoverished non-applicants. Parents appear to be much more 
motivated to seek educational options for younger children, as evidenced by the fact that students 
entering kindergarten or first grade were much more likely to apply for the program and students 
entering junior high or high school were much less likely to apply. 

When comparing applicants to income-eligible non-applicants within relevant grade 
bands, we see that the test-score advantage of applicants relative to non-applicants is limited to 
younger elementary students and is not present among either the junior high or high school grade 
bands. In fact, junior high students appear to be statistically similar to eligible non-applicants 
regarding educational and demographic characteristics and high school applicants only differ 
from comparable non-applicants in that they are more likely to be African American. This 
largely confirms the results of previous studies that have found that low-income African 
American students are more likely to apply for voucher programs when given the opportunity to 
do so. 



To the extent that an element of selectivity has emerged in the OSP it has been at the 
stage of scholarship use. Student race, primary language, and family income do not distinguish 

Campbell, West, and Peterson, “Participation in a National, Means-Tested School Voucher Program...”; Howell, 
“Dynamic Selection Effects in Means-Tested, Urban Voucher Programs...” 
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users from non-users, but some educational characteristics do set users apart from non-users. 
Scholarship users are much more likely to be at the early stage of their education, K-4, and users 
at that stage evidence reading and math test scores that are moderately higher than non-users. 
Older students who use the scholarship have initial test scores that are statistically similar to non- 
users. However, students with a learning or physical disability were much less likely to use a 
scholarship in the first year of program implementation - the single largest distinction between 
the two groups. 

It is impossible to know at this point if the sizable difference between scholarship users 
and non-users regarding learning and physical disabilities is the result of parental decisions 
regarding what is best for their child or limitations in the ability of participating private schools 
to accommodate the special educational challenges that some scholarship students face. The 
District of Columbia Public Schools has a large “private placement” program for students with 
special education needs that sent 2,595 students to private or suburban public schools at public 
expense in 2004, at an average tuition cost of $27,575 per student.^^ 

When District parents decide on their own to enroll their special needs child in a private 
school, the child’s Individual Education Plan and the resources attached to it do not transfer to 
the parentally chosen private school. For parents of students with a moderate-to-severe 
educational disability, the $7,500 voucher may provide insufficient funds, in their view, to obtain 
the array of educational services necessary to educate their special needs child. Although two- 
thirds of the private schools participating in the OSP in the first year reported that they accept 
students with learning disabilities, the overwhelming majority of such schools fully include such 
students in their standard educational environment, with no separate special education program. 

www.dcschoolsearch.dc.gov/schools/report results. asp?report id=14/ (accessed November 11, 2004). 

Wolf et al, “Evaluation of the DC Opportunity Scholarship Program. . pp. 14-15. 
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Parents may plausibly be deciding that a public school environment, with an extensive array of 
support services, is better for their special needs child than a private school environment with no 
special programs, in spite of the award of the voucher. 

There is substantial additional evidence, though preliminary, that scholarship non-users 
may largely be declining to use the scholarship due to the special programs and more extensive 
facilities at their existing schools. Non-users were more likely than users to report that their 
child’s school offered special programs for both disadvantaged and advantaged students, and a 
more extensive physical plant including a gym, cafeteria, and library. Many parents offered a 
scholarship might have compared their child’s existing school to the schools available to them in 
the program and decided that their current school was the better option. Although the families of 
scholarship users were slightly more advantaged in terms of mother’s education, residential 
stability, and parental involvement, scholarship non-users appear to be much more advantaged in 
terms of the features and programs of the schools that they previously attended. 

All of these findings should be taken with a grain of salt. The initial cohort of eligible 
applicants to the OSP was modest in size, numbering only 1,848 from both public and private 
schools. The second cohort of applicants was slightly larger, including 2,199 eligible applicants. 
The characteristics, motivations, preferences, and experiences of this second cohort of applicants 
could be decidedly different from the original group, painting a contrasting picture of who 
chooses and uses an OSP scholarship. Moreover, with a larger and richer set of data regarding 
OSP applicants, non-applicants, users, and non-users, in the future we will be able to conduct 
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more sophisticated statistical analyses of the factors that predict the decisions to apply for the 
OSP and use the scholarship once offered. 

For now, our initial analysis suggests that disadvantaged students and families are 
applying for the first ever federal school voucher program. Among scholarship recipients, 
somewhat educationally advantaged students are more likely to use the scholarship and students 
attending schools with more educational programs and more extensive facilities are more likely 
to decline to use the scholarship when offered. 



See, for example, the nested logit analysis employed by Campbell, West, and Peterson, “Participation in a 
National, Means-Tested School Voucher Program...”, and the survival models estimated in Howell, “Dynamic 
Selection Effects. . .” 
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